Automatic Aesthetic Photo-Rating System

Growing prevalence of smartphone makes photography easier than ever. However, the quality of photos varies widely. Because judging the aesthetic of photos is based on several "rule-of-thumb", it remains difficult for computers to rate photos without manual intervention. In this work, we utilize aesthetic features of photos and machine learning techniques to automatically distinguish good photos from bad ones. Our system is able to achieve 10-fold cross-validation rate of 82.38%. We believe this technique forms the basis of various novel applications, including real time view-finding suggestion, automatic photo quality enhancement, and massive photo rating. INTRODUCTION Rating image aesthetic, as observed in [3] [4], is a very challenging problem. The difficulties are manifold. First, determining image quality remains very subjective. Abundant experience is necessary for being a professional photographer, and there is no effective way to digitalize those rules-of-thumb. Second, the same photo, if viewed by different people with different aesthetic accomplishment, might receive contradicting scores. There lacks consistent principles to classify photos based on their quality. To solve this problem, we need a universal representation of those photography rules, and teach computers to discern good photos from bad ones. Automatic rating is important because it forms the ground stone of various novel applications useful in multiple stages of digital imaging. Applications spanning from creation, post-processing, and social sharing, are all based on this technique. For example, intelligent camera could have realtime suggestions built into the view-finder, letting the user know where to point and shoot. It would be far greater than simply showing a 3-by-3 grid without any active suggestion, as shown in Figure 1. Also, post-processing software can automatically determine the best way to enhance photos without any manual intervention. Furthermore, if equipped with this technology, social websites like Facebook and Flickr would be able to recommend great photos more frequently than photos with poor-quality. In short, we see a high demand in automatic photo-rating that has the potential to make photography friendlier and more intelligent. In this work, we picked multiple aesthetic features and modeled them as simple and intuitive features. These features were trained using automatic classifiers such as random forest, SVM and Bayes network. Finally, a model is generated to predict the aesthetics class of any photo. Figure 2 shows the framework of the overall system. We collect a dataset of 1942 images from DPChallenge, a photograph contest website [1], where people submitted photos to be rated by the public. One advantage of adopting this photo database is that these photos have been quantitatively scored from 1 to 10 by a large set of users. We collect 1000 top rated images with average rate between 7.4 to 8.6 points as the high quality photos, and 942 lowest rated images that are scored between 1.8 to 3.2 points as low quality photos. The rest of this paper is organized as follows. First, we introduce aesthetic features used in the model. The training methods are presented thereafter. Finally, experimental results are illustrated and discussed. AESTHETIC FEATURES To design features representing photo quality, we determine the perceptual criteria that people used to judge photos. We reference principles of photography and select several important criteria used by professional photographers to improve photo quality. In our system, we need saliency map as a way to segment object and determine area of interest. We adopt the saliency map proposed by [12], which is fast and robust. Figure 1. An example of passive suggestion, showing a 3-by-3 grid on an iPhone when user takes a photo.